Output 2022-06-06 22:03:27

This manuscript creates a computationally reproducible workflow that generates the following figures and interactive documents from our single github repository:

BOM data

Australia’s National Bureau of Weather (BOM).

Open weather data

This is another option to the BOM data and provides a series of variables with similar attributes as the BOM API. In future these sections of code maybe helpful with generation of automatic conversion from a single format to another….

Reformatting data

First we need to join the information by the same date and time.

Date

Total precipitation

Total precipitation can be worked with in several ways. From real rain radar to predictions of rain in the 3hour period (not sure if it is before or after at this point (jUne 2022)).

1 day accumulation satellite data from noaa
## character(0)

Prediction

Multiple different organisations provide predictions of the Canberra weather. Here I have combined these predictions into a general estimate of weather from each forecast for different time incriments. To do this the coding needs to take each prediction and collate the relevant information into a structure that allows us to compare different versions of the weather prediction along a timeline of Australian times and dates….ACT times and dates sorry….

## 
## Attaching package: 'maps'
## The following object is masked from 'package:purrr':
## 
##     map

Radar

#access new data
source(here::here("R/additional_code/bomrangRADAR.R"))
## file downloaded to:data/BOMradarIMG/radar.png
#visualise images stored
#xx
# get_radar_imagery(product_id = "IDR403")

Current Canberra radar BOM

Introduction

Here we have taken advantage of the tools and other available tools for open science to create a dynamic dashboard for a local landscaping company. The outcome of the project is a simple interactive dashboard including mapping and predicting of the future weather from several different data sources. The ability to then extend this work into a commercially viable structure is still a long way away. To demonstrate this we have created a reproducible workflow that is curated in RMarkdown and published to github for free to share. With the development of these open source tools we hope to make it apparent that more application within the R and particular the RMarkdown environment to help facilitate both small business growth and scientific discovery.

Examples

Although the tools we have demonstrated here apply to simple, small scale (non-viral) applications. There are many examples of these sorts of applications publicly (cite). This can only indicate the number also applied in private. If this is the case the increase advantages of these tools may result in only sections of the social and global community having access to such sources. There is evidence of this in

include_graphics(here::here("graphics/weatherMap.png"))
## Warning in include_graphics(here::here("graphics/weatherMap.png")): It is highly
## recommended to use relative paths for images. You had absolute paths: "D:/
## Weather2022/graphics/weatherMap.png"

Study area

The study area for this data is the ACT region. We have divided this area up into unique areas that the landscaping business does regular jobs. This allows us to generate locally specific information with some understanding of the confidence of the evidence based on the proximity of the location to the weather station and forecasting information. Figure 1 represents the areas and divisions we have used to group our data as we update the information using our R code at regular intervals.

Methods

The approach to estimating and developing models for real-time data estimation and prediction has many applications and developments. Here we combine methods from the biological, computational and social science fields to make the most of the data aval;iable at the same time as accounting for bias in the collection, visualisation and communication of this information. We choose to use weather as a example as the field is developing fast with the additional data from satellites and other devices in communities allowing for much better response times in dangerous situations such as war and natural disasters.

Data collection

We collect weather data from the following sources:

  • BOM
  • Open weather
  • xx
#package info current check
# BOM
# OWM_API_KEY
# Sys.setenv(OWM_API_KEY = OWM_API_KEY)

library(owmr)

current <- get_current("Canberra", units = "metric", lang = "en")
class(current)
## [1] "list"         "owmr_weather"
forecast <- get_forecast("Canberra", units = "metric")
# class(forecast)

cbr <- search_city_list("Canberra")

current_multiple <- find_cities_by_geo_point(
  cbr$lat,
  cbr$lon,
  cnt = 5,
  units = "metric"
)
# class(current_multiple)

Dataset generated

We then use functions and tidy data wrangling tools (dplyr etc) to generate a standard weather dataset for this flexdashboard and application.

#create visdiagram
knitr::include_graphics("graphics/weatherdataFIG-page-0.png")

This will then allow the product to be passed to the landscaping company for testing and further development.

Images collected

The below section deals with data collection from image data such as satillites and gif’s from web pages. We collect image data from the following sources:

Statistical model

Supporting code/documentation

  • tidyverse
  • dataspice
  • BOM
#latest dataset
tab3 <- read.csv("https://raw.githubusercontent.com/green-striped-gecko/covid_canberra/main/data/last.csv")


# Aggregate method

cols <- c( "red", "yellow","blue")

addBuses <- FALSE

tab3$Contact <- factor(tab3$Contact,levels=c(  "Close"  , "Casual", "Monitor") )
cc <- as.numeric( tab3$Contact)
ncols <- c("black","cyan")
nn <- as.numeric(factor(tab3$Status))
nn2 <- ifelse(nn==1,nn, 3)



tab3$Date <- substr(tab3$Date,1,10) #ditch day of the week
tab3$Date <- as.Date(tab3$Date, format = "%d/%m/%Y")
labs <- paste(tab3$Contact, tab3$Status,tab3$Exposure.Location, tab3$Street, tab3$Suburb, tab3$Date,tab3$Arrival.Time, tab3$Departure.Time, tab3$doubles, sep="<br/>") 

###############################################
##plot the map
df_shared <- crosstalk::SharedData$new(tab3[,c(1,8,2:7)])


m <- leaflet(df_shared) %>% addTiles()
m
#map of stations and data sources
# library()

But by adding additional data sources we are able to increase the number of weather stations reporting to this dataset using:

  • owmr

Results

This report generates a collection of images and data needed to distribute and inform userrs of the data and updates.

Variables

Our curated dataset allows for the aims of our project meet and have resulted in the following baseline set of variables. To create the dataset above there are several important structural choices that have been made based on the overall project goals and the other avaliable tools.

# table of metadata about variables

Setup

First of all you have to store your API key in an environment variable called OWM_API_KEY. For the current session this can be done with:

OWM_API_KEY
Sys.setenv(OWM_API_KEY = OWM_API_KEY)

Get data

Due to the ... parameter in R you can pass all parameters defined in the openweathermap-api-reference to the functions fetching data.

library(owmr)

current <- get_current("Canberra", units = "metric", lang = "en")
class(current)
## [1] "list"         "owmr_weather"
forecast <- get_forecast("Canberra", units = "metric")
# class(forecast)

cbr <- search_city_list("Canberra")

current_multiple <- find_cities_by_geo_point(
  cbr$lat,
  cbr$lon,
  cnt = 5,
  units = "metric"
)
class(current_multiple)
## [1] "list"      "owmr_find"
# bbox_greater_london <- c(-0.489, 51.28, 0.236, 51.686, 10)
# current_greater_london <- find_cities_by_bbox(bbox_greater_london)
# class(current_greater_london)
# current_greater_cbr %>% owmr_as_tibble() %>%
#   .[, c(columns, "name")]

Get similar-ish data frames with owmr_as_tibble

columns <- c("dt_txt", "temp","pressure", "humidity" ,"temp_min", "temp_max")

current %>% owmr_as_tibble() %>%
  .[, columns]
forecast %>% owmr_as_tibble() %>%
  .[1:4, columns]
current_multiple %>% owmr_as_tibble() %>%
  .[, c(columns, "name")]

Actual temperature

Leaflet integration

cbr <- search_city_list("Canberra")

# Get data for 9 cities around cbr
owm_data <- find_cities_by_geo_point(
  lat = cbr$lat,
  lon = cbr$lon,
  cnt = 9,
  units = "metric"
) %>% owmr_as_tibble()

owm_data %<>% parse_columns(list(temp = round))
head(owm_data)
# Create a popup template
popup_tpl <- paste0(
  "<b>{{name}}</b></br>",
  "{{coord_lon}}, {{coord_lat}}</br>",
  "{{temp}}°C, ",
  "<i>{{weather_description}}</i>"
)

# Test it ...
popup_tpl %$$% owm_data %>% head(2)
## [1] "<b>Canberra</b></br>149.1281, -35.2835</br>6°C, <i>broken clouds</i>"            
## [2] "<b>Russell</b></br>149.151, -35.2977</br>5°C, <i>light intensity shower rain</i>"
library(leaflet)

leaflet(width = "100%") %>%
  addProviderTiles("CartoDB.DarkMatter") %>%
  add_weather(owm_data, icon = owm_data$weather_icon, template = popup_tpl)
##snapshot weather

Furthermore, you can add weather map layers from OpenWeatherMap to the map object via add_owm_tiles. For available layers see owm_layers.

Communication

From static to dynamic documents and back again using rmarkdown and knitr…..

To combine the data for this project the following functions have been developed:

Discussion

Coming soon…

Canberra only

Canberra is:

  • small
  • BOM has only three official weather stations that report to the BOM database. These are shown in Figure 1.
  • Divide areas based on workflow. This is obviously dynamic in many cases but this is where it can be changed.
Application

Landscaping and garden works are common locally based small businesses that are heavily impacted by local weather conditions. We use this case study to create a dynamic dashboard of curated locations and there association weather data for a morning (7am report). For a single live site see xx [coming soon]

to reproduce dashboard run the following code…

Weather icons

I will create a set of unique weather icons in the future. For now

owm_data$weather_icon
## [1] "04n" "09n" "09n" "09n" "09n" "04n" "04n" "04n" "09n"

Historical data

  • kaggle?

References

References should follow the ACM standard. The example provided here uses the class file and bib style file. For example, we could write that published a review on EDM in this journal; other reviews were published later . The provided ref.bib file contains examples of virtually every possible citation type.

Working with this project

Appendices

Supplementary technical material (e.g., mathematical proofs,descriptions of experimental procedures and additional code and notes).

Supplementary material should be collected in an appendix at the end of the paper (before the acknowledgements and the references sections).

A1: Setup and build current repository

A2: RMarkdown help and support

Figures and tables

Each figure and table must be mentioned in the text, and must be numbered consecutively in order of appearance (with captions in lower case). For the review process, the figures should be integrated into the text rather than being inserted at the end of the document.

As JEDM is an electronic journal, authors are encouraged to use colours when it enhances visibility and understanding of figures.

# source("R/openLeafMap.R")
# leaf1

Examples of a figure and a table are given in Figure~\(\ref{tab:fig1}\) and Table~\(\ref{tab:1}\).

For LaTeX, a style file named is provided on the journal’s web site. An MS Word file containing this example text is also provided, , which can be used as a template (see the Styles menu). Note that copy/pasting into the template may not ensure correct formatting but selecting a region of text and applying the appropriate style should always work.

Footnotes and acknowledgments

Footnotes should be used sparingly and indicated by consecutive superscript numbers in the text. Material to be footnoted should appear at the bottom of the page on which it is referenced. Acknowledgments and grant numbers should be put into a separate ‘Acknowledgment’ section right before the list of references.

Articles per Day

Comments per Day

Spam per Day

References

Supporting materials (from journal template)
Page numbering and sectioning
Sections and subsections
Subsection levels
Fourth level.

Submission

Bookdown

Manual referencing
 

Support our work here:

anthony.davidson@canberra.edu.au

R + RMarkdown + Github.